Noise-Robust Speech Recognition Technologies in Mobile Environments

In mobile environments, user input and operation via voice are simple and effective. However, the speech recognition performance is highly influenced by ambient noise in the mobile environments, which may cause a significant deterioration of the performance, and there is a strong demand for improvement of the performance. In order to resolve these issues, we conducted research in two directions: “advancement of multi-modal speech recognition” and “noise processing technologies.” 1) Research Related to Advancement of Multi-Modal Speech Recognition a) Multi-modal speech recognition using side-face moving image data We propose a multi-modal speech recognition method that uses side face and lip moving image data as part of a noise-robust speech recognition method in mobile environments. This method uses side face image data and allows the user to input voice in a natural posture. Figure 1 shows a configuration of the multimodal speech recognition proposed. In this method, acoustic and image data are merged using a multi-stream Hidden Markov Model (HMM), to improve the recognition performance. b) Examination of stream-weight optimization method in multi-modal speech recognition In the multi-modal speech recognition of acoustic and image information using the aforementioned multi-stream HMM, we propose to optimize a normalized likelihood criterion, and confirm that the error ratio can be reduced by approximately 40% when the volume of sample data is small compared to the conventional likelihood-ratio maximization method. 2) Research on Noise Processing Technologies for Speech Recognition The signal input to the recognition system is continuous without any decisive information about end-points. Techniques for automatically recognizing continuous speech signal are necessary. For this reason, we propose a methodology to automatically and robustly detect utterance intervals under conditions where the Signal to Noise Ratio (SNR) changes over time, based on a tree-structured noise overlay speech model, and confirm that the proposed method does improve the speech recognition performance.